NPS55-84-006 


NAVAL  POSTGRADUATE  SCHOOL 

Monterey,  California 


A  NOTE  ON  THE  DERIVATION  OF  THEORETICAL 
AUTOCOVARIANCES  FOR  ARMA  MODELS 

by 

Ed  McKenzie 


February  1984 


FedDocs 

D    208.14/2 

NPS-55-84-006 


Approved  for  public  release;   distribution  unlimited 

Prepared  for: 

Naval  Postgraduate  School 

Monterey,  California  93943 


{j  xo*,  vu\a:  DPS-ss-su-0O(* 

NAVAL  POSTGRADUATE  SCHOOL 
Monterey,  California 

Commodore  R.  H.  Shumaker  David  A.  Schrady 

Superintendent  Provost 

This  work  was  supported  by  the  Naval  Postgraduate  School  Foundation 
Research  Program  under  contract  with  the  National  Research  Council. 

Reproduction  of  all  or  part  of  this  report  is  authorized. 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (Whan  Data  Enttrmd; 


DUDLEY  KNOX  LIBRARY 

NAVAL  POSTGRADUATE  SCHOOL 

MONTEREY  CA  93943-51 01 


REPORT  DOCUMENTATION  PAGE 


read  Dfrmucnoifs 

BEFORE  COMPLETING  FORM 


1.    REPORT  NUMBER 

NPS55-84-006 


2.  GOVT  ACCESSION  NO 


J.     RECIPIENT7!  CATALOG  NUMBER 


4.     TITLE  (and  Subtitle) 


A  NOTE  ON  THE  DERIVATION  OF  THEORETICAL 
AUTOCOVARIANCES  FOR  ARMA  MODELS 


S.    TYPE  OP  REPORT  ft  PERIOO  COVERED 

Technical 


7.     AUTHORf»J 

Ed  McKenzie 


»      PERFORMING  ORO.  REPORT  NUMBER 
■  .    CONTRACT  OR  GRANT  NUMBER/tJ 


«0.    PROORAM  ELEMENT.  PROJECT.  TASK 
AREA  *  WORK  UNIT  NUMBERS 

61152N;    RROOO-01-100 
N0001483WR30104 


9.     PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

Naval  Postgraduate  School 
Monterey,  CA  93943 


II.     CONTROLLING  OFFICE  NAME  AND  AOORESS 


12.    REPORT  OATS 

February  1984 


IS.    NUMBER  OP  PAQES 


14.    MONITORING  AGENCY  NAME  ft   ADDRESS^!/  dlflmrmnt  from  Controlling  Olllca) 


IS.    SECURITY  CLASS.  (•!  «*•«•  rwMtj 

Unclassified 
Tiic 

SCH 


IS*.    OECLASSIFICATION/OOWNORAOINO 
"    EOULE 


16.     DISTRIBUTION  STATEMENT  (of  thla  Rmport) 

Approved  for  public  release;  distribution  unlimited 


17.     DISTRIBUTION  STATEMENT  (of  tha  abattact  antarad  In  Block  20,  II  dlttarmtt  from  Kap+rt) 


18.     SUPPLEMENTARY  NOTES 


19.     KEY  WORDS  (Conflnum  on  ravaraa  old*  II  nacaaaary  and  Identify  by  block  mtmbat) 

Theoretical   ACVS 

Stationarity 

ARMA  likelihood  function 


20.     ABSTRACT  (Contlnua  on  ravaraa  alda  It  nacaaamwy  and  Idantlty  by  block  w— km) 

Derivation  of  the  theoretical  autocovariances  of  an  ARMA  model  is  important 
for  a  number  of  purposes  associated  with  the  estimation  and  testing  of  the 
model.  One  common  algorithm,  due  to  McLeod  (1975),  involves  solving  a  system 
of  linear  equations.  By  deriving  the  determinant  of  the  matrix  of  coeffi- 
cients in  these  equations  we  can  ascertain  the  behaviour  of  the  algorithm  with 
respect  to  the  stationarity  of  the  ARMA  model. 


DD  ,5 


FORM 
AN  73 


1473  EDITION  OP  I  NOV  68  IS  OBSOLETE 

S/N  0102-  LF-  014-  6601 


UNCLASSIFIED 

SECURITY  CLASSIFICATION  OP  TNIS  PAOt  (Whm  D*taT*frmt) 


SECURITY  CLASSIFICATION  OF  THIS  PACE  (Whun  Datm  Enffd) 


S/N  0102-  LF- 014-6601 


SECURITY  CLASSIFICATION  OF  THIS  PAGEfWhwi  Dmf  Enfrmd) 


SUMMARY 

Derivation  of  the  theoretical  autocovariances  of  an  ARMA  model  is 
important  for  a  number  of  purposes  associated  with  the  estimation  and  testing 
of  the  model.  One  common  algorithm,  due  to  McLeod  (1975),  involves  solving  a 
system  of  linear  equations.  By  deriving  the  determinant  of  the  matrix  of 
coefficients  in  these  equations  we  can  ascertain  the  behaviour  of  the  algorithm 
with  respect  to  the  stationarity  of  the  ARMA  model. 

ACKNOWLEDGMENT 

I  gratefully  acknowledge  the  support  of  a  National  Research  Council 
Associateship  at  the  Naval  Postgraduate  School  in  Monterey,  California,  where 
this  work  was  carried  out. 


A  NOTE  ON  THE  DERIVATION  OF  THEORETICAL 
AUTOCOVARIANCES  FOR  ARMA  MODELS 


Ed  McKenzie 

Department  of  Operations  Research 

Naval  Postgraduate  School 

Monterey,  California  93943 

and 

Department  of  Mathematics 

University  of  Strathclyde 

Glasgow,  Scotland 


February  1984 


McLeod  (1975,  1977)  presents  a  method  for  deriving  the  theoretical  auto- 
covariance  function  of  an  ARMA  model.  He  notes  its  uses  in  simulating  ARMA 
processes  and  in  deriving  the  asymptotic  distributions  of  the  estimated  auto- 
correlations. The  procedure  can  also  be  used  in  deriving  ARMA  model  residuals 
(Ansley  and  Newbold,  1979);  in  obtaining  the  asymptotic  distributions  of 
parameter  estimates  and  residual  autocorrelations  (McLeod,  1978);  and  in 
calculating  the  exact  likelihood  function  of  the  Gaussian  ARMA  model  (Ljung 
and  Box,  1979;  Ansley,  1979;  and  Dent,  1977).  Ansley  (1980)  and  Ansley  and 
Kohn  (1982)  have  also  extended  McLeod's  algorithm  to  vector  ARMA  models. 

An  alternative  and  computationally  superior  procedure  to  McLeod's  in  the 
univariate  case  has  been  proposed  by  Wilson  (1979).  It  also  has  the  advantage 
that  the  stationarity  of  the  process  may  be  tested  directly  within  the 
algorithm  generating  the  autocovariances.  The  procedure  for  maximum  likeli- 
hood estimation  proposed  by  Dent  (1977)  also  incorporates  a  test  of  station- 
arity, as  do  some  others  in  that  a  Cholesky  decomposition  of  the  generated 
covariance  matrix  is  later  derived.  However,  this  is  not  general. 

The  purpose  of  this  note  is  to  examine  the  behaviour  of  the  McLeod 
algorithm  with  respect  to  stationarity. 

Consider  the  ARMA(p,q)  process  {X.}  given  by 

Xt  "  *lXt-l  "  "•  "  *pXt-p  =  dt  "  Vt-1  "  *••  "  Vt-q  * 

If  p  =  0  ,  the  process  is  always  stationary.  If  p  >  0  the  process  is 
stationary  if  and  only  if  the  roots  of  the  polynomial  equation 


i  vp"k  =  u  (2) 

k=0 

all  lie  within  the  unit  circle.  For  later  reference,  denote  the  roots  of  (2) 

P    k 
by  z  ,z  ,...,z  ,  and  denote  the  polynomial   V  <j>  z   by  <|)(z)  . 

1  2    p  k=o  k 


If  p  >  0  the  variance  and  the  first  r  autocovariances  (Yn»Y, ,...,y  ), 
where  r  =  max(p,q),  are  obtained  by  solving  a  system  of  linear  equations. 
The  matrix  of  coefficients  of  these  equations,  A  say,  is  given  in  McLeod 
(1975)  and  Ljung  and  Box  (1979).  Our  interest  is  in  how  the  stationarity  of 
(1)  affects  the  solution  of  these  equations.  We  determine  this  by  expressing 
|A|,  the  determinant  of  A  ,  in  terms  of  the  roots  of  (2). 

Consider  the  matrix  A(t)  obtained  by  replacing  <j>.  by  <j>.t   in  A  . 
The  i   row  of  A(t)  may  be  expressed  as  the  sum  of  two  row  vectors,  thus: 

—j   ~   '^-j-i^   , . . .  ,(})q  ,0 . .  .0) 

+  (0,<f>.jt  ,...  ,<|>  t  ,0,...0)  ,  i  =  l,2,...,r+l  . 

Clearly,  £._1  =  <j>(t),*for  i  =  l,2,...,r+l,  where  1_   is  the  unit  vector 

( 1,1,...  ,1) ' .  Hence,  <j>(t)  is  an  eigenvalue  of  A(t)  and  so  a  factor  of 

|A(t)|.  Similarly,  <j>(-t)  is  a  factor  of  |A(t)|.  We  may  note  for  later  that 

♦  UM-t)  =  I     (l-z2t2)  . 

1  =  1  p 

Suppose  z(*±l)  is  any  solution  of  (2),  and  £=   (l,z,...,z  )  .  We  can 

p+1_i        i  -i 
use  (2)  to  write  a.t  in  the  form       <j>.  .  .(zJ-z  J)  ,  i  =  l,2,...,r+l  . 

j-1   J"i+1     , 

Note  that  if  r  =  p  ,  a  5  =  0  .  Suppose  now  that  z    is  also  a  solution  of 

-1     -r 

(2),  and  £-  =  (l,z  ,...,z  )'  .  Then,  A  (_£+£.)  =  0  which  is  possible  if 

and  only  if  A  is  singular.  Thus,  (1-z.z.)  is  a  factor  of  |A|.  Further, 
if  cj).  is  replaced  by  A.t   in  (2)  the  roots  become  {tz.  :  i  =  l,2,...,p}  , 

and  we  can  deduce  that  (1-z.z. t  )  is  a  factor  of  |A(t)|.  Thus, 

P   P         7 
P(t)  =  n   n  (1-z.z. t2)  is  a  factor  of  |A(t)|. 

i=l  j=i     7  J 
Note  that  P(t)  is  a  polynomial  of  degree  p(p+l)  in  t  .  If  r  =  p  , 

the  (k,p+2-k)th  element  of  A(t)  has  the  form  (<j>  tp  +  terms  of  lower  power) 

for  k  =  l,2,...,p+l  .  Thus,  |A(t) |  is  also  a  polynomial  of  degree  p(p+l)  . 

If  r  =  q  then  A(t)  has  the  form 


A(t)  = 


Ap(t)    0 


B(t)    L(t) 


where  L(t)  is  a  lower  triangular  matrix  with  units  along  the  main  diagonal. 
Thus,   |A(t)|  =  |A  (t)|   and  A  (t)  is  (p+1)  x  (p+1)  and  has  the  property 
ascribed  to  A(t)  when  r  =  p  .  Hence,  in  both  cases,  |A(t) |  is  a  polynomial 
in  t  of  degree  p(p+l)  .  Further,  |A(0) |  =  1  =  P(0)  .  Thus,  taking  t  =  1, 
we  have  shown  that 


P   P 
|A|  -  n  n  (l-z.z  ) 
i=l  j=i      J 


(3] 


where  z-.z^,...,!   are  the  solutions  of  (2). 


CONCLUSIONS 

From  (3)  it  is  clear  that  |A|  =  0  if  and  only  if 
either    (i)  there  is  a  root  on  the  unit  circle; 

or      (ii)  there  are  a  pair  of  roots  symmetric  about  the  unit  circle,  i.e. 
z  and  z" 

It  is  comforting  to  know  that  the  procedure  will  fail  and  no  auto- 
covariances will  be  generated  when  the  process  is  non-stationary  for  either  of 
the  reasons  given.  On  the  other  hand,  it  is  clear  that  a  non-stationary 
process  which  does  not  satisfy  either  (i)  or  (ii)  will  yield  a  set  of  "auto- 
covariances".  It  may  be  possible  to  detect  this  at  once,  e.g.  yn  may  be 
negative  or  less  than  y.      in  magnitude  for  some  k  .  In  general,  however, 
these  values  can  be  shown  to  be  spurious  only  by  showing  that  the  correspond- 
ing covariance  matrix  is  not  positive  definite. 

This  may  be  achieved  in  a  routine  manner  within  the  overall  procedure. 
Dent  (1977)  suggests  a  check  on  the  singular  value  decomposition  of  the  co- 
variance  matrix  and  Pagano  (1973)  discusses  a  modified  Cholesky  decomposition. 
However,  if  the  purpose  of  the  procedure  is  estimation  we  may  have  to  generate 
autocovariances  from  different  sets  of  parameters  a  large  number  of  times.  In 
such  a  case  an  algorithm  such  as  that  proposed  by  Wilson,  which  checks 
stationarity  while  it  generates  autocovariances,  would  clearly  be  preferable. 
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